ON THE REPRODUCING KERNEL OF A PONTRYAGIN SPACE 
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Abstract. Wg give necessary and sufficient conditions under which the repro- 
ducing kernel of a Pontryagin space of d X 1 vector polynomials is determined 
by a generalized Nevanlinna pair o{ d X d matrix polynomials. 



1. Introduction 

1.1. By Baire's category theorem, the Pontryagin space *B in the title is neces- 
sarily finite dimensional (see Remark 12.11 below) and hence is a reproducing kernel 
space. Indeed, if (*B, [• , ■]'s) is an ?i-dimensional Pontryagin space of d x 1 vec- 
tor polynomials and if B{z) is a x n matrix polynomial whose columns Bk{z), 
k S {!,... ,Ti}, form a basis of 58, then the reproducing kernel of *B is the d x d 
matrix polynomial in z and w* given by 

K{z,w) = B{z)G-^B{wy, z,weC, 

where G is the n x n Gram matrix associated with B{z), that is, 

G = [9]k]lk=i, 9'jk = [Bk,Bj\<s,, j, fc e {1, . . . , n} 

(see [31 Example 2.1.8] and the remark following it). The reproducing kernel of a 
reproducing kernel space is unique but can often be written in various ways. In 
this paper we give necessary and sufficient conditions under which K[z,'w) above 
is a polynomial Nevanlinna kernel. This means that it can be written in the form 

r^, X . N M[z)N{w)* ~N[z)M{w)* ^ . , 

K{z,w) = Km n{z,w) :^ — — — — — — z,w&C, z ^ w* , 

z — w* 

where M{z) and N{z) arc d x d matrix polynomials such that 

M{z)N{z*)* - N{z)M{z*y = for all z e C 

and 

(1.1) rank[M(z) N{z)\ = d for at least one z e C. 

If, in addition, the equality in p.ip holds for all z e C, then the Nevanlinna kernel 
Km,n{z, w) is called a full Nevanlinna kernel. 

The following theorem is the main result in this paper. It is proved in Section 31 
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Theorem 1.1. Let ^ be a [finite dimensional) Pontryagin space of d x I vec- 
tor polynomials. Denote by S<b the operator of multiplication by the independent 
variable in 58 and by Eq, the operator of evaluation at a point a g C. Then the re- 
producing kernel of S is a polynomial Nevanlinna kernel if and only if the following 
two conditions hold: 

(A) The operator S<s is symmetric in *B. 

(B) For some a G C we have ran(S'(jj — a) = *B H kcr Ea- 

In this case the reproducing Nevanlinna kernel is full if and only if the equality in 
dU holds for allaeC. 

We think Theorem 11.11 is new, possibly even in the positive definite case, that 
is, the case where the space *B is a reproducing kernel Hilbert space of vector 
polynomials. In that case *B in the theorem is a special case of L. de Branges' Hilbert 
spaces of entire functions. For scalar functions, see [H]; for vector functions, see [13] 
and [2]. In particular, [TU Theorems 1-3] are closely related to Theorem ll.il For 
results on the indefinite scalar case we refer to the series of papers on Pontryagin 
spaces of entire functions by M. Kaltenback and H. Woracek. More specifically, [26l 
Theorem 5.3] is closely related to Theorem 11.11 with d = 1, [27l Proposition 2.8] 
can be used to obtain a scalar version of Theorem 11.21 below, and [Ml Lemma 6.4] 
is linked with Theorem 15.11 in Section [5] The emphasis in this paper is on vector 
polynomials and an indefinite setting. 

1.2. In the proof of Theorem 1 1.1 1 we use the following result which shows that the 
condition ([B|) in Theorem 11.11 completely determines the structure of *B as a linear 
space. We believe Theorem 11.21 is also new, but closely related to results around 
[201 Proposition 2.3]. For the proof of Theorem 11.21 we refer to Section [31 

Theorem 1.2. Let 58 be a finite dimensional linear space ofdx 1 vector polynomials 
and let a G C The equality 

(1.2) ran(S'<8 - a) = S n kerSa 

holds if and only if there exist nonnegative integers ^i, . . . , fid and a d x d matrix 
polynomial W{z) with dct VI^(q;) 7^ such that the space *B consists of all vector 

polynomials of the form W{z) [^1(2) • • ■ Pd{z)] where Pj{z) runs through all scalar 
polynomials of degree strictly less than fj.j, j G {1, . . . , d}. The matrix W{z) can be 
chosen such that 

(1.3) {a e C : dctW{a) ^ 0} = {a e C : ran(S'<B - a) = <Bnkcr£;„}. 

It follows that the dimension of 58 in Theorem 1 1.2 1 is //i + • • • + /i^. If the conditions 
(fS]) and ((B| of Thcorcm ll.ll hold. then the numbers ^1 , . . . , /x^ are the Forney indices 
of the block matrix polynomial [M(z) N{z)^ corresponding to the reproducing 
Nevanlinna kernel of 58. Moreover, the defect numbers of S<s coincide with the 
cardinality of the set {j & {1, . . . ,d} : /Zj > O}, sec Remarks 14.11 and 14.31 This 
offers a direct way of determining the dimension of the reproducing kernel space 
5B with reproducing Nevanlinna kernel Km,n{z,w) and the defect numbers of S<s 
from the block matrix polynomial [M(z) N{z)\ . 

In the scalar case {d—\) the space 58 in the above theorems is analogous to the 
so-called Szcgo space, in the Hilbert space setting defined and studied in [34l [35] 
and in the Pontryagin space setting in [1] . In the literature there are many papers 
characterizing special forms of the reproducing kernel of a reproducing kernel space. 
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Of those related to a reproducing kernel Pontryagin space we mention [3 Section 6] 
and [2]. We refer to the references in these papers for papers dealing with the 
Hilbert space case. The characterizations in these works arc often in terms of a 
special identity to be satisfied by the difference-quotient operator on the space. In 
some cases, such as in [2l Theorem 4.1] and [121 Problems 51, Theorem 23] the 
invertibility of A'(z, z) for some values of z plays a role in proving the asserted 
representation of the kernel K{z,w). We give in Section [6] some examples where 
dctK{z,w) = for all z,w eC, see Example 16.61 and Example 16.71 

1.3. A pair {M{z), N{z)} oi dxd matrix functions M (z) and N{z) is called a gen- 
eralized Nevanlinna pair if the functions are meromorphic on C \ M, the intersection 
of the domains of holomorphy hol(M) of M{z) and hol(iV) of N{z) is symmetric 
with respect to the real axis, 

(1.4) M{z)N{z*)* - N{z)M{z*y = for all z G hol(A/) n hol(iV), 

(1.5) rank[M(z) N{z)] = d for at least one z e hol(A/) n hol(iV), 
and the Nevanlinna kernel 

/ N M{z)N{w)*-N{z)M{w)* , , , , 

(1.6) Km,n{z,w) := — z — ^J^*^ — ■^'^ ^ hol(Af)nhol(7V),z 7^ w* , 

has a finite number of negative squares. Here, by a finite number of negative 
squares we mean that the set of numbers of negative eigenvalues counted according 
to multiplicity of the self-adjoint matrices of the form 

[X*KM,N{ZJ,Z^)X^]'^^^^^ 

with 

n e N, Xi € C^, Zi G hol(M) n hol(7V), z, ^ z* , ij G {1, . . . ,n} 

has a maximum. If this maximum is k, then we say that the pair and the kernel have 
K negative squares. If k = the adjective "generalized" is omitted; in that case the 
matrix functions are holomorphic at least on C\M. The number of positive squares 
is defined in the same way. The pair and kernel are called full if the equality in (|1.5p 
holds for aU z G hol(M) n hol(Af). If a (generalized) NevanUnna pair {M{z), N{z)} 
is such that N{z) = Id, the dxd identity matrix, then it is identified with its first 
entry M{z) and M(z) is a (generalized) Nevanlinna function. 

Nevanlinna pairs and generalized Nevanlinna pairs have been used in interpola- 
tion and moment problems (see [3n|, [IJ [S| and [5]), the description of generalized 
resolvents (see |28| ) and in the theory of boundary value problems with eigenvalue 
dependent boundary conditions (see [17l[T^, [15] and [9]). Theorem 1 1.1 1 arose in our 
study [lOj of an eigenvalue problem for an ordinary differential operator in a Hilbert 
space with boundary conditions which depend polynomially on the eigenvalue pa- 
rameter. In that paper we linearize the original problem by extending the Hilbert 
space with a finite dimensional Pontryagin space of d x 1 vector polynomials. This 
paper concerns the structure of such spaces. 

1.4. The Nevanlinna pair in a Nevanlinna kernel is not unique (see the paragraph 
before Examplc l6.7p and if {M (z) , N (z)} is a pair that determines the kernel, then 
the polynomial matrix N{z) may be such that det7V(z) = for all z G C. In 
Section [5] we prove that one can always choose the pair so that detA''(z) ^ and 
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the rational generalized Nevanlinna matrix function N{z)^^M{z) is essentially a Q- 
function of the symmetric operator S<b- We show that every self-adjoint extension 
of StQ with nonempty resolvent set gives rise to a reproducing Nevanlinna kernel for 
the space *B. The proof of Theorem 11.11 given in Section |4] is geometric, the proof 
of the first if statement in Theorem 11.11 given in Section [5] is analytic. The last two 
examples in Section |6l Example 16.61 and Example 16.71 also serve to show that this 
analytic proof is constructive. In Section |6] we present three corollaries of Theorem 
11.11 and four examples. 

In Section [2] we fix the notation related to vector and matrix polynomials and 
we recall the Smith normal form and the Forney indices of a matrix polynomial. 
Moreover, we prove some lemmas on the structure of a degenerate subspace of 
a finite dimensional Pontryagin space, the defect numbers of a simple symmetric 
relation in such a space and on polynomial Hcrmitian kernels. Although most 
proofs in this paper are based on methods from linear algebra, in the sequel we 
assume that the reader is familiar with (i) Pontryagin spaces and (multi-valued) 
operators on such spaces such as symmetric and self-adjoint relations (as in [24j . 
[I9] and [H]), (ii) generalized Nevanlinna matrix functions (as in [30l|3T]) and (iii) 
reproducing kernel Pontryagin spaces (as in Chapter 1] and [31 Chapter 7]). 

The notion of a Q-function of a simple symmetric operator in a Pontryagin space 
is recalled in Section [S] 

Acknowledgements. The authors thank Prof. Marius van der Put for many 
discussions about Theorem 13.41 and another proof of it and Prof. Daniel Alpay for 
acquainting them with his unpublished note [1] which among other things lead to 
Remark 12.11 and pointing out the references [34l ES] . We also thank the referee for 
useful comments. 

2. Notation and basic objects 

2.1. The symbols N, R, and C denote the sets of positive integers, real numbers 
and complex numbers. For d E N the vector space of all d x 1 vectors is written as 
and Id stands for the dx d identity matrix. The k-th row of Id will be denoted 
by ed,k- For k G {1, . . . ,n} the subspace of spanned by e^.i, . . . , ed,k will be 
called a top coordinate subspace o/C^; it will be denoted by <Cf. The corresponding 
dx d projection matrix is denoted by Pd.k- We consider Cq = {0} a top coordinate 
subspace spanned by the empty set. 

By C^[z] we denote the vector space over C of all polynomials with coefficients 
in C^. The space is identified with the subspace of all constant polynomials in 
C^iz]. If d = 1 we simply write C[z] and C. For / € ^'[z] \ {0} with 

f{z) — ao + aiz + • • • + a„z" 

and for the zero polynomial we define 

dcg / = maxjfc G {0, . . . , n} : ak ^ 0} and deg = — oo. 

Matrix polynomials are written as B{z)^M{z),N{z), . . ., that is, with their argu- 
ment z\ we use the bold face P(2:), 8(2;), . . ., for d x 2c? matrix polynomials. Vector 
polynomials are sometimes written with and sometimes without their argument. 
The Fraktur alphabet 21, *8,£, Jo, ... is used to denote vector subspaces of ^'[z]. 
One exception to this is that £ will be used for a subspace of C^''[z]. An inner 
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product on S is denoted by [ • , • ] <8 . In a vector space, the symbol © denotes the 
direct sum of subspaces. 

Remark 2.1. A Banach space with a countable Hamel basis is separable and 
hence, by [32|, it is finite dimensional. Since {z" : n S {0}UN} is a countable 
Hamel basis of C[z], the space C^lz] and all its subspaces also have countable 
Hamel bases. Therefore any Pontryagin subspace of C^lz] is finite dimensional. In 
spite of this fact, to emphasize the finite dimensionality, we continue to speak of 
finite dimensional Pontryagin subspaces of C''[2:]. 

We introduce some special subspaces of C''[z]. Let n e {0} U N. The symbol 
C''[z]<„ stands for the set of all / G C^[z] such that deg/ < n. In particular, 
C''[z]<i = and C'^[z]<o = {0}. A subspace £ of C^lz] is called canonical if there 
exist nonnegativc integers /ife, fc G {1, . . . , d}, such that 

d 

= {[pi{z) ■ ■ ■Pd{z)]'^ ■■ Pk{z) £ C[z], degpk < Mfc, k £{!,.. . 

The numbers fii, . . . , fi^ will be called the degrees of £. Without loss of generality 
we can assume that they are ordered: /ii > • ■ ■ > /id > 0. Then a canonical 
subspace is uniquely determined by its degrees. Clearly, the dimension of £ is the 
sum of its degrees. 

Next we introduce some useful operators on C''[z]. By Pd^k, k G {1, . . . ,d}, we 
denote the natural extension of Pd^k to ^'[z], by S : C^[z] — > C^[z] the operator 
of multiplication by the independent variable, that is, 

{Sf){z) = zf{z), f G C%], 

and by Ea : C^[z] the evaluation operator at the point a G C: 

It follows from the fundamental theorem of algebra that 
(2.1) ran(S'-a) ^kerSc,- 

A wide class of operators on C^[z] is induced by dx d matrix polynomials. If M{z) 
is such a polynomial we define the operator M : C^[z] — >■ C^[z] by 

{Mf){z)^M{z)fiz), .feC%]. 

Clearly, MS = SM . A square matrix polynomial is unimodular if its determinant is 
identically equal to a nonzero constant. If M{z) is a unimodular matrix polynomial 
we will call M a unimodular operator. In this case M is a bijection and its inverse 
is also a unimodular operator. 

2.2. In the sequel we use that any nonzero d x n matrix polynomial B{z) admits 
a Smith normal form representation (see for example [221 Satz 6.3] or [25]): 

'D{z) 





(2.2) B{z) = U{z) 



V{z), 



where U{z) is a d x d unimodular matrix polynomial, V{z) is an n x n unimod- 
ular matrix polynomial and the matrix in the middle is a d x n matrix in which, 
for some I G {1, . • . ,min{d, n}}, D{z) is a diagonal I x / matrix polynomial with 
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monic diagonal entries: D{z) = dia.g(bi{z), . . . , bi{z)^ such that bi{z) is divisible by 
i £ {1,...,/— 1}. Notice that ranki?(a) = / if and only if 61(0;) 7^ 0. If 
for some z G C the rank of B(z) is d (ri, respectively), then I = d {I = n) and the 
zero block row (column) in the matrix in the middle of the right hand side in (|2.2p 
is not present. 



Remark 2.2. The matrix in the middle of the right hand side in ()2.2p is uniquely 
determined by B{z). In this paper B[z) often is a matrix polynomial whose columns 
form a basis of a subspace 58 of C''[z]. Then for any d x n matrix polynomial 
Bi(z) whose columns also form a basis of *8, the middle term of its Smith normal 
form is identical to that of B{z). Thus, the number I and the monic polynomials 
bj{z),j G {!,..., I}, above are uniquely determined by the subspace *8 of C'^[z]. 

2.3. Let 8(2) he a. d X 2d polynomial matrix. For j £ {!,..., d} let be the 
degree of the j-th row of 8(2). By definition, a degree of a row is the degree of its 
transpose. Define Soo, the internal degree and the external degree of S(z) by: 

- —1 ... " 



Soo = lim 

■ ■■■ z' 
extdeg S(z) = ai + ■ ■ ■ + ad, and 



S(^), 



intdegS(z) = max{ degm(z) : m(z) is a d x d minor of 8(2:)}. 
For a proof of the following theorem we refer to j33) . 

Theorem 2.3. Let P(z) be a d x 2d matrix polynomial with rankP(z) — d for all 
z €z <C. Let S(z) be a matrix polynomial in the family 

(2.3) {;7(2)P(2) : U{z) unimodular). 
The following statements are equivalent: 

(a) extdeg 8(2) = minjextdeg J7(2)P(2) : U{z) unimodular^. 

(b) rank8oo = d. 

(c) extdeg 8(2) = intdeg8(2). 

(d) 8(2*)* has the "predictable degree property": 

For every u{z) = [^1(2) • • • Ud{z)\^ S 'C'^[z] we have 

deg(8(2*)*u(2)) = maxjo-j + dcgUj(2), j e {1, . . . , d}}. 

A matrix polynomial 8(2) in the family (|2.3p satisfying the conditions (jaj)-([d| is 
called row reduced. The multiset {cri, . . . , ad} of row degrees for each row reduced 
matrix in the family (|2.3|) is the same. Its elements are called the Forney indices 
of any of the matrices in the family <\2.Z\ . in particular of P(2). We extend this 
definition to the case where the d x 2d matrix polynomial P(2) has full rank for 
some 2 S C. For that we use the following lemma which is a standard tool in system 
theory, see for example [2T| . 

Lemma 2.4. Let P(2) be a dx 2d matrix polynomial with rankP(2) = d for some 
2 e C. Then 'P{z) admits the factorization: 

(2.4) P(2) = G(2)T(2) for all 2 e C, 

where G{z) is a d x d matrix polynomial with det G{z) ^ and T{z) is a d x 2d 
matrix polynomial with rankT(2) = d for all 2 G C. This factorization is essentially 
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unique, meaning that if also P{z) ~ Gi(z)Ti(z) for all z G C, where Gi{z) and 
Ti(z) have the same properties as G{z) and T(z), then for some unimodular d x d 
matrix polynomial E{z): Gi{z) = G{z)E{z)~'^ and Ti{z) = E{z)T{z), zeC. 

The Forney indices of P(z) in the lemma are by definition the Forney indices 
of the matrix polynomial T(z) in the factorization (|2.4p . By the second part of 
the lemma, this definition is independent of the choice of the matrix G{z) in this 
factorization. 

For convenience of the reader we give a proof of Lemma 12.41 based on the Smith 
normal form of a matrix polynomial. 

Proof of Lemma [2.41 Let F{z) have the Smith normal form (j2.2p . The assumptions 
imply that I = d and that the matrix in the middle of (|2.2p is equal to [D{z) O] . 
Set G(z) = U{z)D{z) and T{z) = [id O] V{z). Then the factorization ([2^1) 
holds and G(z) and T(z) have the properties mentioned in the lemma. To prove 
uniqueness we use the fact that, since T(z) and Ti{z) have full rank for all z £ C, 
they have right inverses, see [25j . These are 2d x d matrix polynomials S(z) and 
Si{z) such that T(z)S(2) = Id and Ti(z)Si(z) = Id for all z € C. Define the 
matrix polynomials E{z) = Ti(z)S(z) and F{z) = T(z)Si(z). Then the equality 
G(z)T(z) = Gi(z)Ti(z) implies E{z) = Giiz)-^Giz) and F{z) = G(z)-iGi(z), 
hence E{z)F{z) ~ Id for all but finitely many z € C. By continuity the last equality 
holds for all z g C, hence E{z) is unimodular and has the stated properties. □ 

2.4. The next two lemmas concern finite dimensional Pontryagin spaces. By the 
positive (negative) index of a Pontryagin space /C we mean the dimension of a 
maximal positive (negative) subspace of /C; evidently, the dimension of JC is equal 
to the sum of the indices. 

Lemma 2.5. let K, he a Pontryagin space with positive and negative index equal to 
n. let C be a subspace of IC with dim£ = 2n — t. If C contains a maximal neutral 
subspace of K, then is the isotropic part of C and CjC^ is a Pontryagin space 
with positive and negative index equal to n ~ t. 

Proof. Let A/" be a maximal neutral subspace contained in £. Since = A/", the 
inclusion A/" C £, yields C JV C C. Therefore, £^ is the isotropic part of £ and 
dim£'^ = T. Let £ = £^ + £_ + be a pseudo-fundamental decomposition of £. 
Since A/" is a neutral subspace of £, we have n = dim A/" < r + dim£-|-. Therefore 

2n ~ T ~ dim £ = r + dim £_ + dim £^ > t + n — t + n — t — 2n — t. 

This proves that dim£_ = dim£_|- = n — t. □ 

Recall that a symmetric relation 5 in a Pontryagin space IC is simple if S has no 
non-real eigenvalues and K, ~ span{ker(S'* — z) : z e C \ E}. Below mulS"* stands 
for the multi-valued part of the adjoint S* of S: mulS"* = {g G JC : {0,17} € S*}. 

Lemma 2.6. Let S be a simple symmetric relation in a finite dimensional Pontrya- 
gin space of dimension n. Then the spaces mul 5*, ker S* , and S* D zl, z S C, have 
the same dimension d' , say. In particular, the defect numbers of S are both equal 
to d' . Furthermore, dim ranS* — dim S = dim domS* ~ n ~ d' and dim S* = n + d' . 

Proof. First notice that by [SI Proposition 2.4] S is an operator and S has no 
eigenvalues. The following statements are equivalent: 



8 



B. CURGUS AND A. DIJKSMA 



(a) dini(mulS'*) = fi'. 

(b) codim (dom 5*) = d! . 

(c) codim(ran(5 — z*)) = d' for all z S C. 

(d) dim(S'* n zl) = d' for all z e C. 

The relation (domS*)^ = mills'* implies the equivalence (jaj)<^(|b|. The equivalence 
(Ib|<^(jcj) follows from the fact that S — z* is one-to-one. By taking the orthogonal 
complements we obtain the equivalence (jcj)<;=>([d|. Notice that fdl with z = implies 
that d' = dim(ker S"*) . The equalities n — d' = dimdomS* = dimS = dimranS* 
follow from (|b| and the fact that S is an injcctive operator. Since dim S* = 2n — 
dim S the last equality follows. □ 

2.5. A dx d matrix function K(z, w) will be called a polynomial Hermitian kernel 
if it is a polynomial of two variables z and w* and K(z,w)* — K{w,z), z,w £ C 
This implies that the degree of K{z,w) as a polynomial in z equals the degree of 
K{z,w) as a polynomial in w* . If we denote this common degree by p — 1, then 
K{z, w) can be expanded as 

p— 1 p— 1 

(2.5) K(z,u;) = ^^AJfcZ■'"«;*^ z,w&C, 

j=0 k=Q 

where Aj^^j, k S {0, . . .,p ~ 1}, are d x d matrices. Since K{z,w) is a Hermitian 
kernel, the dp x dp block matrix 



(2.6) A = 



Aqo ■ ■ ■ ^0,p- 



Ap-1.0 ■■■ A. 



p— 1 



is self-adjoint. It also follows that the number of negative squares of K{z, w) equals 
the number of negative eigenvalues of A and the number of positive squares of 
K(z, w) equals the number of positive eigenvalues of A. The dimension of the 
reproducing kernel space corresponding to K{z, w) is the rank of A. These obser- 
vations are used in the proof of the following lemma. 

Lemma 2.7. Let K{z, w) be a dx d matrix polynomial Hermitian kernel of degree 
p — 1. For q Cz N set 

Lg{z,w) = i{z'^ -w*'^)K{z,w), z,weC. 

If Q ^ P' then the positive and the negative index of the reproducing kernel Pon- 
tryagin space with kernel Lq{z,w) are equal and coincide with the dimension of the 
reproducing kernel Pontryagin space with kernel K{z,w). 

Proof. Write K{z,w) in the form (|2.5p and denote by A the matrix (|2.6p . We 
calculate the coefficients of the matrix polynomial Lq{z, w) for q > p: 

p—ip—i p—ip—i 
Lg{z, w) = Ajkz^w*'' - \w*'i ^ ^ A^kz^w*^ 

j=o k=a j=o k=o 

j=0 k=0 j=0 k=0 
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q+p—1 p-1 



p-1 q+p-1 



J2 J2^'^u-i)k^^w*'' 

j=a k=o 

q+p-l q+p-1 

j=a k=o 



j=0 k=0 



where we set Ajk = whenever j < or A: < or j > p — 1 or /c > p — 1. In other 
words, the 2d{p + g) x 2d{p + q) self-adjoint matrix formed by the coefficients of 
Lq{z^ w) is given by 

" -\A 
B = 
lA 

where the in the center is a d(q — p) x d{q — p) matrix. With 





E ^ ^ 



I 

^75 



Idp 

ildp 



^d(q-p-) 




ildp 


Idp 



we have EE* 



Id(q+p) and 



E*BE = E* 









-lA 




'A 

















E = 











lA 
















-A 



Therefore the rank of B is twice the rank of A. Moreover, B has equal numbers 
of positive and negative eigenvalues. Since the positive and negative index of the 
reproducing kernel Pontryagin space with kernel Lq{z, w) coincide with the number 
of positive and negative eigenvalues of B the lemma is proved. □ 

A polynomial reproducing Nevanlinna kernel introduced in the Introduction is a 
polynomial Hcrmitian kernel. Since in the proof of Thcorcm ll. li the polynomials in a 
Nevanlinna pair never appear separate we adopt the following equivalent definition 
of a polynomial Nevanlinna kernel: A d x d matrix function K(z,w) is called a 
polynomial Nevanlinna kernel if it can be represented as 



(2.7) 



F{z)Q-^P{w)* ^i{z-w*)Kiz,w) for ah z,w e 



where Q is a 2(i x 2(i self-adjoint matrix with d positive and d negative eigenvalues 
and P{z) is a c? X 2d matrix polynomial such that P{z) has rank d for some z <E C. 
With 



(2.8) 



Q = Qi 











and P(z) = [M{z) N{z)] 



the definition in the Introduction is obtained from the new one. The assumptions 
on Q imply that there exists a constant invertible matrix T such that Q = TQiT*. 
Now, if we write P{z)T = [M{z) iV(z)], we have K{z,w) = Km,n{z,w). Since 
F{z) is a polynomial, the condition that rankP(2:) = d for some z S C implies that 
rankP(z) = d for all but finitely many z € C. A polynomial Nevanlinna kernel will 
be called a full Nevanlinna kernel if P(z) can be chosen such that rankP(z) — d 
for all z G C. 
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3. Proof of Theorem 11.21 

3.1. Let S be a vector subspace of C''[z]. By S<3 we denote the range restriction 
of 5 to 58, that is, 

domS'.B = » n S'-i'B, {S<sf){z) = zf{z), f G domS-o,. 

In graph notation this means: 

S<s = : /,.9 e OS, giz) = zf{z) for aU z e C}. 

By (|2.ip . for a G C wc have 

(3.1) ran(S'<8 - a) = (5" - a) (*B n S'^^) C ran(S' - a) n *B = «B n ker£;„. 

The reverse inclusion is equivalent to the implication 

/ e a G C, /(a) = =4> /(z) = (z — Q;)g(z) for some g&domS<s- 

In some cases this implication does not hold. For example, it does not hold for any 
a G C in the space 05 C C^[z] given by 

,2 



05 



gq + a2Z 
bo + biz 



■ ao, 02, 6o, bi e 



Indeed, 05 contains [z^ — z — a]^ which is at z = a, but S docs not 

contain [z + a l]^. That the implication, or equivalently, equality in (j3.1l) holds, 
is characterized in terms of canonical subspaces of C^[z] in Theorem 11.21 in the 
Introduction. This section is devoted to the proof of this theorem. 

Let _B(z) be a d X n matrix polynomial whose columns form a basis for 05, 
n = dim*B. Then, as will be shown in the proof of Theorem II. 2 [ the sets in (|1.3p 
are equal to {a G C : bi{a) ^ O}, where 6i(z) is the scalar polynomial in the 
Smith normal form (|2.2p of B{z). We will first prove Theorem 11.21 for the case 
where the sets in (|1.3p are equal to C, see Theorem 13.41 below. In this case W{z) is 
unimodular. The proof of Theorem 13.41 is based on the following three lemmas. 



Lemma 3.1. Let ^ be a finite dimensional subspace of C^lz] such that 

ran(S'f8 — a) = 05 H keri^Q for all a £ C 
IfdomS^ C 05' C 05, then ran(S'av - a) = n ker£;„ for all a e C. 

Proof Let / G 05' n kcr E^. Then / G S n kcrE:^ = ran(S'<B - a), that is, 
f ^ Sg — ag for some g G domS'tg C 05'. From f,gG^' we infer g, Sg G 05'. Hence 
g G domS'fg' and / = {S<s' — ct)g. This proves *8' n ker Ea C ran(S'f8' — aj. Since 
the reverse inclusion is obvious, the lemma is proved. □ 

Lemma 3.2. Let 58 be an n-dimensional subspace o/C'^[z]. Then 

(3.2) » n ker£:„ = {0} for all a G C 

if and only if there exists a unimodular operator W such that 58 = W^y^, where CjJ 
is a top coordinate subspace ofC^. 

Proof. If n = 0, the statements are trivial with W{z) = Id- From now on we assume 
n > 1. If B{z) is any d x n matrix polynomial whose columns form a basis of *B, 
then, clearly, 

(3.3) {a G C : rankS(a) = n} = {a G C : 5B n ker£;„ = {0}}. 
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Assume (|3.2p . Let B(z) he a d x n matrix polynomial whose columns form a 
basis of *B. By p.3|) . for all a G C the rank of B{a) is n and n < d. Hence B{z) 
admits the Smith normal form (see (|2.2p ): B{z) = U{z) [/„ O] V{z)^ where U{z) 
and V{z) are unimodular. Define 



(3.4) W{z) = U{z) 



V{z) 
Id-. 



Then W{z) is a unimodular dxd matrix polynomial and from B{z) = W{z) [/„ O] ^ 
it follows that *B = W<C^. This proves the only if statement. 

To prove the if statement, assume that there exists a d x d unimodular matrix 
polynomial W{z) such that *B = WC^, where CJ^ is a top coordinate subspace of 
C*. Then the columns of B{z) = W{z) [/„ O] ^ form a basis of *8 and the rank of 
B{a) is n for all a e C. The equality ([321) follows from ([331). □ 

Lemma 3.3. Let *8 be an n- dimensional subspace ofC^lz] and let <t be a canonical 
subspace of C'^[z] with degrees /^i > • ■ ■ > /id > o/ which k are positive. Assume 
£ + 5£ C <B and 

(3.5) <B n ker C C + S-e: for all a e C. 

Then there exists a unimodular operator W which acts as the identity on £ + 5£ 
and is such that 

(3.6) ^^W{Ct + *t + S(t), 

where m = n — (/ii + • • • + /-t(i)(> 0). 

Notice that CjJ^ + C + 5£ is a canonical subspace. If m < /c, then CJ^j + £ + 5£ 
coincides with £ + S'C and W — Id- 



Proof. If £ = {0} the statement follows from Lemma [321 From now on we assume 
£ 7^ {0}. Then > 0, consequently k G {1, . . . ,d} and Cf C £. We consider two 
cases: k ~ d and k < d. 

(i) Assume k = d. Then C'' C £ C <B. Let / e *8. It can be written as 
f{z) = f{0) + zh{z) = f{0) + {Sh){z). Then Sh = /-/(O) G <B. Since {Sh){0) = 0, 
by dSH) we get ^/i e «8 n ker C £ + S'C, which implies / = /(O) + She€ + S<t. 
That is, *8 = £ + 5£. In this case m = d and with W = Id the lemma is proved. 

(ii) Assume k < d. If £ + S'C = *8, then ([31]) holds with = and m = fc, 
implying that Cf„ C £. From now on we assume that £ + S€ is a proper subspace 
of *B. Recall that Pd^k is the coordinate projection. A trivial, but important 
observation is 

(3.7) Ea (£ + 5£) = = ran Pa.k for all a e C. 

Let a S C be arbitrary and let / e *B be such that {Id — Pd,k)f{ct) — 0. By p.7p . 
there exists a p e £ + 5£ such that p{a) ~ Pd.kfia), hence 

(/ - P){a) = {Id - PdM)f{a) + Pd.kfia) - p{a) = 0, 

that is, / - p e keri?a. Since also f - p e ([331) implies f - p e € + 5£. Thus 
both p and / — p belong to £ + S'£, implying that / e £ + S'£. We have proved the 
implication: 

(3.8) / e aeC and (/<i - Pd,k)f{a) = ^ / £ £ + ^£. 
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Let £o be a subspace of *B be such that 

{€ + Se)n£o^ {0} and <8 = (£ + ^C) © -Co- 
The dimension of £o is 

j = n - (/ii H h /id + fc) > 1- 

Let Bo{z) be a d X J matrix polynomial whose columns form a basis of £o- Decom- 
pose B(){z) as 

^0,6(2:) 

where Bo^tiz) is a fc x j matrix polynomial and Bo^b{z) a {d — k) x j matrix poly- 
nomial. We will prove that 

(3.9) ranki?o,fe(Q;) = rank(/(j — Pd,k)Bo{a) = j for all a e C. 

The first equality is trivial. To prove the second let a € C be arbitrary and x £ <C^ 
be such that {la - Pd,k)Bo{a)x = 0. Set f{z) = Bo{z)x. Then / G -Co and 
{Id - Pd,k)f{o) = 0. By dilH]), / e (£ -h SC) n -Co, consequently / = 0, that is, 
Bo{z)x = for all z € C. Since the columns oi Bo{z) form a basis of -Co, this implies 
a; = 0. This proves p.9p . Hence j < d—k. li j = d— k, the {d—k)x {d— k) matrix 
polynomial Wb{z) := Bofi{z) is unimodular. li j < d ~ k we can extend Bo, 6(2) to 
a unimodular {d — k) x {d — k) matrix polynomial (also denoted by) Wb[z) with 
det Wb{a.) ^ in the same way as the matrix B{z) was extended to W{z) in ()3.4p 
by the means of the Smith normal form. In both cases the first j columns of Wb{z) 
are the columns of Bo^b{z)- 

Let Wt{z) be the fc x {d~ k) matrix obtained from the k x j matrix BQ,t{z) by 
adding d ^ k j zero columns on the right. Define the d x d matrix polynomial 
W{z) by 

h Wt{z) 
0{d-k)xk Wb{z)_ 

Then W{z) is unimodular and W{z)ed,k+i,l = 1,---,J, are the columns of the 
matrix Bo{z). The operator W acts as the identity on€ + S€ and M^Cf„ = C^ + -Co, 
where 

m = k + j = k + n- [^i + • • • + /i^ + fc) = n - (/^i + • • • + /i^) . 
Hence M^(Cl + £ + S'C) = + £0 + £ + S'C = <B. □ 
Theorem 3.4. Let *B he a finite dimensional subspace ofC^[z]. The equality 

(3.10) ran(S'<8 -a) ^Snker-Ba for all aeC 

holds if and only if there exist a d x d unimodular matrix polynomial W{z) and a 
canonical subspace £ of C^[z] such that S = WC 

Proof. We first prove the if statement. To prove (|3.10p it suffices to show that 

S n ker£;„ C ran(S'<8 - a). 

Let / e S n kerE'c. Then /(a) = and / = Wg for some g e €. Since W is 
unimodular, g{a) ~ 0. Since £ is canonical, the polynomial g{z)/{z — a) belongs 
to £. Therefore f(z)l{z - a) = W{z){g{z)l{z - a)) G *B, hence / e ran(S'(s - a). 

We prove the only if statement by induction on the dimension of Assume 
(|3.10|) . The theorem is obviously true if dimS = 0. Lemma (3.21 implies that it 



W{z) 
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is true if dini*B = 1 for then *B n i?Q = {0}. Let n G N and state the inductive 
hypothesis: 

If 21 is a subspace of C^[z] with dim 21 < n and such that 

(3.11) ran(S'a - a) = 21 n kcr for all a e C, 

then there exists a unimodular d x d matrix polynomial operator F{z) such that 
i^2t is a canonical subspace of C'^[z]. 

Let 05 be a finite dimensional subspace of C^[z] such that (|3.10p holds and 
dim*B — n. Then 21 = domS'fg is a proper subspace of *B. Therefore dim 21 < n. 
If 21 = {0}; then *B n kcr Ea = ran(S'i8 — aj ~ {0} and the theorem follows from 
Lcmma [3?2l Now we assume 21 7^ {0}. By Lcmma lSTTl thc subspace 21 satisfies (|3.11|1 . 
By the inductive hypothesis there exists a unimodular matrix polynomial F{z) such 
that 2) := i^2l is a canonical subspace of ^^[0]. Since F and S commute we have 
D = F2t = FdoinS<s = dom Sf<b, hence 2) + S*® C C/<B. To apply Lemma to 
F^ we need to verify ([331). Let / e «B be such that (F/)(a) = 0. Then f{a) = 
and, by (|3.10p . there exists age dom S'tg = 2t such that / = S<sg — a,g G 2t + 521. 
Therefore, Ff e D + SD, which verifies ([331) . Lemma lO applied to yields that 
there exists a unimodular operator U such that [/^^F*B is a canonical subspace of 
C'lz]. This proves the theorem with W ^ F-'^U □ 

3.2. The following lemma will be used to deduce Theorem 1 1 . 2 1 from Theorem l3.4l 

Lemma 3.5. Let *B be an n-dimensional subspace o/C'*[2] and let B{z) be a dx n 
matrix polynomial whose columns form a basis of Let I be the size of the square 
diagonal matrix in the Smith normal form (j2.2|l of B{z). Then 

(3.12) {a e C : ran(S'<8 - a) = n kcr Ea] = {a G C : rankS(a) = l] 

if and only if the set on the left hand side is nonempty. In this case dimranS'tg = 
dimS-Z. 

Proof. The only if statement follows from the fact that the set on the right hand 
side in (I3.12p is nonempty. Before proving the if statement we show 

(3.13) dimran5'<B < dim*B — I. 

For all a G C we have ran(S'f8 — a) C *B n ker Ea, and hence 

dimranS'is = dimran(S'iB — a) < dim(S n ker Ea) = dim*B — rank_B(a). 

Consequently, I = maxcgc ra'iik-B(a) < dim*8 — dimranS'tg. This proves p.l3|) . 

To prove the if statement assume that ao G C is in the set on the left hand side 
of p.l2p . Then equality holds in p.l3p . Indeed, this follows from 

dim *B — / > dim ran 5*135 

= dimran(S'gs — ao) 

= dim(<Bnkcr£'a„) 

= dim*B — ranki?(ao) 

> dim<B - I. 

This proves the last statement in the lemma. Now the equality p.l2p follows from 
the following sequence of equivalences which hold for all a G C: 

rankB(a) = Z ^ dim(<B n kcriJ^) dim*8 - / 

dim(*B n ker Ea ) = dim ran S<s 
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□ 



^ dim(S n ker Ea) = dimran(5<8 — a) 
<^ Ta.n(S<B - a) ^ ^ nkcr Ea- 

Proof of Theorem W .2\ Wc first prove the if statement. It suffices to prove the 
inclusion *B H kcv Ea C ran(S'(B — a), as the reverse inclusion always holds. Let 
/ e «8 n kcr£;„. Then /(a) = and / = VFg with 5 G £. Since W{a) is 
invertible, g{a) — Q. As £ is canonical, the polynomial h{z) = g{z)/{z — a) belongs 
to £. Therefore W{z)h{z) e *8 and (x — a)W{z)h{z) ~ f{z), which implies / G 
ran(S'(8 — a). 

To prove the only if statement, assume that (|1.2p holds for a ~ ao. Let B{z) be 
a, d X n matrix polynomial whose columns form a basis of *B. Let 

\D{z) 



B{z) - Uiz) 







be the Smith normal form (|2.2p of -B(z) where 15(2;) is an Z x Z diagonal matrix with 
nonzero diagonal entries. Now define the space Si C C^[z] as the span over C of 
the columns of 



Bi(z) = C/(z) 







Viz). 



Set 



F{z) = U{z) 



U{z) 



D{z) 
h-i\ 

Then 05 = F*Bi and deti^(Q;o) 7^ 0. Moreover, since 

{a G C : dctF(a) 7^ 0} = {a G C : rankB(a) = 1} 

and by Lemma [3.51 (|l-3p holds for F(z). From detF(Q:o) 7^ it follows that 

ran(S'(8 — ao) = *B n kcri?„„ ran(S'(Bj — ao) = Si n kcr E^o- 

Since rank_Bi(a) = ? for all a G C, Lemma l3.5l implies that 

ran(S'i8j^ — a) — Si n kcv E^ for all a G C. 

By Theorem 13.41 there exists a unimodular matrix U{z) such that £ = C/^^Si is 
a canonical subspace of C''[z], hence S = W€. with = FU . Finally, (|1.3p holds, 
because J7 is unimodular and -F(z) satisfies (|1.3[) . □ 



3.3. Theorem 1 1 . 21 can also be formulated in terms of matrix polynomials: 

Theorem 3.6. Let S he an n- dimensional subspace o/C'*[z], n> 1. Let B{z) be 
a d X n matrix polynomial whose columns form a basis of S. Let bi{z) and I be 
as in the Smith normal form (|2.2p of B{z). Then I + dim dom 5*23 = dimS if and 
only if there exist 

(a) a dxd matrix polynomial W{z) whose determinant has the same zeros as bi{z), 

(b) nonnegative integers m and Sq > 61 > ■ ■ ■ > 6m with Sq + ■ ■ ■ + Sm = n and 

(c) an invertible n x n constant matrix T 

such that 



(3.14) 



B{z) = W{z) Ps, Ps,z 



T for all z G C, 



where Pg stands for the dxd matrix: Ps = [is O] 
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Proof. For all a G C we have ker(5'<B - a) C !B n kevEa. For all a G C with 
61(a) ^ we have 

/ + dim dom S'fg = dimran(£'a|(g) + dimran(S'(8 — a) 

< dimran(i?a|2s) + dim(*B n keri^a) 
= dim 

and equality holds if and only if ker(S'(8 — a) = S n keri^Q. 

To prove the only if statement, assume I + dimdomS'© = dimS. Then we can 
apply Theorem 11.21 There exist a matrix polynomial W{z) satisfying (jaj) and a 
canonical subspace £ of C"*[z] such that *8 = W€. Let fii > ■ ■ ■ > i^id be the 
degrees of £. Since n > 1. we have /ii > 1. set m = /ii — 1 and 

5j =#{ie{l,...,d} : fi, > j}, J e {0, . . . ,m}. 

Then the equality in (|b| holds. Since the columns of the matrix [Ps„ ■ ■ ■ Ps^z™-'^ 
form a basis for £, there exists a matrix T satisfying such that p.l4p holds. 

To prove the if statement, we note that (jaj)-(jcj) and (|3.14|) imply that *B = W€ 
with £ as above, and hence Theorem 11.21 can be applied and together with the if 
and only if statement at the beginning of the proof yield that / + dim dom 5'<8 = 
dim<B. □ 

Remark 3.7. Denote by {S^)<s the range restriction of to *B. Then in item (|b| 
of Theorem 13.61 m is the nonnegative integer with 

{0} = dom(5'"+^)25 C dom(^™)<B 

and 

Sj = dimdom(S'^)>8 — dimdom(S'-'"'"^)f8, j G {0, . . . , m}. 
Moreover, if we set (5_i = d, then the numbers 

(3.15) ^fc l + max{j G {-1,0,...,to} : (5j > A:}, G {1, . . . , d}, 

arc the degrees of the canonical space M^~^*B. In the next section we will see that 
if *B C C^[z] is a Pontryagin space which satisfies the conditions (|A| and ([B]) of 
Theorem ll.ll then the numbers (|3.15p are the Forney indices of a matrix polynomial 
P{z) in a representation (|2.7p of the Nevanlinna reproducing kernel K{z,w) of *B; 
see Remark 14.31 

4. Proof of Theorem 11.11 

4.1. We divide the proof of Theorem 11.11 in two parts. In the first part we prove 
the if statements and in the second part we prove the only if statements. In the 
first part wc will need characterizations of the defect numbers of the operator S<s 
of multiplication by the independent variable in the Pontryagin space which are 
collected in the following remark. 

Remark 4.1. Clearly, S<s has no eigenvalues and for any subset O of C containing 
more than d x max{deg/ : / G 05} elements we have r\wenran{S<s — w*) = {0} 
or, equivalently, 

05 = span{ker(S'^ — w) : w G 51}. 

Now assume of Theorem ll.il Then, by the above observations, S<3 is a simple 
symmetric operator and hence its defect numbers coincide and are equal to the 
codimension of ranSis , see Lemma 12.61 It follows from Lemma 13.51 that the defect 



16 



B. CURGUS AND A. DI.IKSMA 



numbers of Sfs are also equal to the integer I introduced in Remark 12.21 Hence 
I E {1, . . . ,mm{d,n}}, where n = dim*B. Now also assume ([B|) of Theorem 11.11 
Then I can be characterized in a different way. Indeed, by Theorem II .21 there exist 
a canonical subspace £ C [z] with degrees /ii > • • • > > and a. d x d matrix 
polynomial W{z) with det W{a) ^ such that S ~ WC Since, by Lemma [2761 we 
have n — I = dimdomS'tg — dimranS'tjj and since multiplication by z and by W{z) 
commute, I is uniquely determined by the inequalities: 

(4.1) Ml > • • • > /i; > 1 and m;+i = /id = 0. 

Proof of the if statements in Theorem W.W Assume ([A| and ([B]). We show that *B 
has a reproducing Nevanlinna kernel in steps (i)-(iv). In step (v) we prove the last 
if statement in the theorem. 

(i) By Theorem 11.21 there exist a canonical subspace £ C C'-[z\ with degrees /ii > 
• • • > /^d > and a d X d matrix polynomial W{z) with det W{a) ^ such that 
*8 = W€. Then, by Remark |4.1[ the defect numbers of the symmetric operator Stg 
are both equal to I, where I is determined by the inequalities ()4.1|) . It follows that 
the elements of *B are of the form: 

(4.2) fiz)e^^f{z) = W{z) ""^^^ , 

where x{z) is an Z x 1 vector polynomial and denotes the zero vector of size 
{d — I) X 1. Let n = dim 05 and let B{z) be a d x n matrix polynomial whose 
columns form a basis of 05. Let G be the corresponding Gram matrix and write the 
reproducing kernel K{z,w) of 05 as K{z,w) = B{z)G~^B{w)* , z,w € C. By dH]), 
this representation implies that for each w G C which belongs to the set in p.l2p 
the columns of K{z, w) span an Z-dimensional subspace of 05, in formula: 

(4.3) dim{/<( • ^'w)x : x G C^} = / whenever it; G {a G C : rank_B(a) = Z}. 

(ii) In the following we use graph notation in the space *B © The operator ^rg is 
identified with its graph in 05© 05 and its adjoint is the orthogonal complement 
of S<s in *B © 05 equipped with the Lagrange inner product 

l{f,9}Ap,q}] = -Kia^ph - If.qh), {/,,9},{p,<?} e os©*B. 

Let w e C, X e and {/, Sf} £ S<b be arbitrary. Then 

[{/, Sf}, {K{; w)x, w*K{; w)x}] = -i{[Sf, K {■ , w)x] ^ ~ [/, w*K{; w)x\ ^) 

= ^\{x*{Sf)(vj)-wx*f{w)) 
= 

and hence {{K{- ,w)x,w* K{- ,w)x} : x C^) Q S*^ r\ {w*I) for aU w & C. Ac- 
cording to the definition of defect number (see [HI p. 369]) and by (|4.3p . it follows 
that for aU w G {a e C \ K : rankB(a) = /} 

(4.4) {{Ki-, w)x, w*K{-, w)x} : x e C^} = n (w*!), 
because for such w's both sets have dimension I. Consider the subspace 

(4.5) £o := spsin{{K{-,w)x,w*K{-,w)x} : u; G C, a; G C^} 
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of S^. Since S<b has no eigenvalues, the generalized von Neumann formula given 
in [9l Theorem 3.7] implies that for d + 1 distinct points wo, ■ ■ ■ ,Wd from the set 
jo; G C : Ima > 0, ra.nk B{a) = /} we have 

d 

5^ = 5<B + 5^5 n {w^i) + J2s^n {wji). 

Combined with (|44| and (|4?5| this yields S<b + 2a 'Z S-s + &o, and hence 

(4.6) S;^ = S<s+2n. 

(iii) Let (Q3i, [ • , • ]<Si) be the reproducing kernel Pontryagin space whose kernel is 

Li{z,w) = i{z — w*)K{z,w), ZjWGC. 

We claim that its positive and negative index are I. To prove the claim we consider 
the operator T : (5^, [•,•]) ^ 'Bi defined by T{{f,g}) = Sf ~ g, {f,g} e 5^, 
and show that it is a partial isometry onto Si with null space kerT = S<s- The last 
equality is easy to verify. That ranT = Si follows from (j4.6p as it implies (with 
m; G C and a; e C^): 

(T({A'(-, ■w)x, w*K{-, 'w)x})){z) ~ {z — w*)K{z, w)x = — iLi(z, ■w)x. 

That T is isometric follows from (|4.6p . the symmetry of S<s and the equalities (with 
w, u G C and x, y G C*): 

[{K{;w)x,w*K{;w)x},{K{;v)y,v*Ki;v)y}] 

= -i(^[w*K{-,w)x,K{-,v)y]^ - [K{-,w)x,v* K{-,v)y]^^ 

~ i(v — w*)y* K{v, w)x 
= y*Li{v,w)x 

= [-iLi{-,w)x,-\Li{-,v)y]^^ 

= T{{K{-,w)x,w*K{-,w)x}),T{{K{-,v)y,v*K{-,v)y}) . 

The claim now follows because (S'^/S'©, | • , • ]) is a Pontryagin space with positive 
and negative index I (see ^ Theorem 2.3(c)]) and T establishes a unitary mapping 
between this space and 58 1. 

(iv) Let -81(2) be a d X 2^ matrix polynomial whose columns form a basis of Si, 
and let Qi be the corresponding 21 x 21 Gram matrix. Then Qi is self-adjoint and, 
by the claim proved in (iii), has I positive and I negative eigenvalues. Let B2{z) be 
the d X 2{d — I) matrix polynomial defined by 





B2{z) = Wiz) 



ld~l 



where the zero matrices are of size Z x (d — I). Define the dx 2d matrix polynomial 
P(2) by P{z) = [Bi{z) B2{z)] and the 2c? x 2d block diagonal matrix Q by 



Q 



Qi 






Q2 



where Q2 







ild-i 




Then Q is self-adjoint and has d positive and d negative eigenvalues. We claim that 
(I) rankP(z) = d for some z G C, and 
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(II) P(z)Q-ip(w)* = i{z - w*)K{z, w) for all z,weC. 

Wc prove dJ): The inclusfon <Bi ^ T{S^) C <B + 5*8 and g^) imply that there 
exists an I X 21 matrix polynomial X(z) such that 



Bi{z) = Wiz) 



X{z) 




where now stands for the {d~l) x 21 zero matrix. The complex number a satisfying 
(|B|) belongs to the sets in p.l2p and (|1.3p and hence 

rankA"(a) = ranki?i(Q;) 

= dimspan|Li(a, z/;)2; : it;SC,xeC''} 

= dimspan|i(a — w*)K{a^ w)x : w e C, a; G C''} 

^ dimspanjA' (a, w)y : w &C,y £ C^} 

= dimi;^® 

= ranki?(a) 

= /. 



The equality 



P(z) = W{2 



X{z) 
Id-l 







implies that rankP(Q:) ~ d. This proves (jl]). We prove (|Tl| : 
P(z)Q^ipH* = Bi(z)Qr'SiH* + B2(z)Q2-iB2M* 

i/d-; 




Li{z,w) + W{z) 
i{z — iy*)iir(z, w) 






■ 




Id-l 


Id-l 





/d-i 
/d-i 



Items (jj) and (jn]) show that K{z, w) is a polynomial Nevanlinna kernel for *B. This 
completes the proof of the if statement. 

(v) If (|B|) holds for all a G C, then, by Theorem I3.4[ W{z) is unimodular and the 
proof of (|l| shows that then rankP(z) = d for all z G C. □ 

4.2. In the proof of the only if statements in Theorem 11.11 we use the following 
lemma. 

Lemma 4.2. Let Q be a self-adjoint 2d x 2d matrix with d positive and d negative 
eigenvalues. Let ^{z) be a d x 2d matrix polynomial such that 

(a) P(z)Q-ip(z*)* for all z e C, 

(b) rankP(z) = d for all z G C, and 

(c) P{z) is row reduced and has row degrees ai, . . . , a^, assumed ordered so that 
f 1 > ■ ■ • > cTrf and (Ti = degP(z) =: p. 

Equip C^'' [z] <p with the inner product 



p-l p^i p^i 

1=0 3=0 3=0 



•>2d 
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and consider the following subspace of C [z]<:p : 

£p = span/ zP-^-''w*''Piwyx : w e C, x e C''^ 



, fe=0 



Then the orthogonal complement of Zp in (C^ [z]<p, [• ,-]q) is 
(4.7) £^ = [f{z) e C^^U, : f{z) = V{z*Yu{z) with u{z) G 

It is the isotropic part of £p and £,p/£,p is a Pantry agin space with positive and 
negative index ai + ■■■ + ad- 
Proof. For an element f{z) ~ J^^Zo^^j^'' ^ 'C^'^[z]<cp the following equivalences 
hold: 

fiz) e £^ ^ ( ^ w*''al\ Q-^P{w)* = for all u; e C, 

\k=0 ) 

^ P(z)Q-V(z) = for ah z e C, 

<^ /(z) = P(z*)*M2 for some G C and aU z e C. 

The last equivalence follows from (jaj) and (|b]). To prove that the vector Uz depends 
polynomially on z wc use that the Smith normal form (|2.2p of V{z) is given by: 
P(z) = J7(z)[/ti 0]y(z), where U{z) and V{z) are unimodular matrices. Then 



f[z) = V{z*yuz - 



lJ{z*Yuz ^uz^ V{z*)-* [h 0] U{z*)-*f{z) 



and the right hand side belongs to C'*[z]. This proves (|4.7p . 

Since P(z*)* has full rank for every z e C, it acts as an injection on C'^[z], 
therefore 

(4.8) dim£^ = dim{u(z) e £%] : dcg(P(z*)*u(z)) < p). 

The number on the right hand side can be expressed in terms of the Forney indices 
of P(z). Indeed, since P(z) is row reduced, it has the "predictable degree property" 
(see Theorem [2l3l): 

deg(P(z*)*u(z)) max{(Tj + degMj(z) : j e {1, . . . , d}}. 
Consequently, the space on the right hand side of in (|4.8|) equals 

{u{z) e <C%] : dcguj(z) <P'Crj, j £ 
whose dimension is dp — (cti + • • • + ad) ■ Hence dim = dp — (tri + • • • + ad) and 

dim£j, ^ dimC^'^[z]<p - dim£p ^ dp + {ai ^ h (Td)- 

To prove the last two statements of the lemma wc apply Lemma 12.51 with n — dp 
and T ^ dp — {ai + ■ ■ ■ + ad). The assumptions about Q in the lemma readily imply 
that C^'^[z]<p is a 2(ip-dimensional Pontryagin space with negative index dp. It 
remains to construct a maximal neutral subspace of C^'*[z]<p which is contained in 
£p. We begin with the subspace ^ ~ rani/, where the operator H : C^[z] — > €^"^[2] 
maps u{z) e C^[z] into the polynomial part of P(1/z*)*m(z). For example, if P(z) 
is written as: 

P(z) ^ Po + zPi + ■ ■ ■ + zPPp, 
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then for /c e {0} U N and x € 



if k < p. 



p 



These formulas imply that is neutral in 
{0} U N and x, y € wc have 

min{A;,m} 



zf'-Pj^z^Pp-jX if p<fc 

2d 



•^]<pj [ ' 1 ■ ]q) • Indeed, for k,m G 



H(z''x),H(z^^y) 



Q 



3=0 



i-\-j—p—l — m+k 

ije{o,...,p} 



and the last expression equals because the assumption (jsj) is equivalent to 
E PjQ'^Pk=0 foi-all ?ie {0,...,2p}. 

j-\-k—n 
],k€{0,...,p} 

Since, by ©, Pq = P(0) has full rank, H is degree preserving and hence injective. 
Therefore, dimi] = dp = (1/2) dim(C^''[2;]<p) and Sj is maximal neutral. 

Define the mapping R : C^'^[z]<p -> C^'^[z]<p by = zP^^f{l/z). Then 

R is unitary with respect to [ • , ■ ]q and hence ^Tl RS^ is also a maximal neutral 
subspace of (C'^'^[z]^p, [ • , • ]q). The proof of the lemma is complete if wc show that 
Vt C £p. For that we consider the polynomials of the form 

2p-l 

J2 z'"w*''x, w e C, x€&. 

k=0 



(4.9) 
From 



z''w*''x 



and 

we obtain 
(4.10) 



5Z yi-k 



2p-l 



2p-l 



2p-l 



z'^w*'' 



k=0 



3=0 
p-1 



k=Q 



2p-l-j 



k=0 



k=0 



k=0 



k—p 



/2p-l 



p-1 



\ k=0 



z''w*^x 



'^^z^w*^'P{w)* X + higher order terms. 



fc=0 



Since the space C'*[z]<2p is spanned by polynomials in (|4.9p . each element of C'*[2]<p 
is also a sum of polynomials in (|4.9p . As H is degree preserving, the polynomials 
in .5 = H{€-'^[z]^p) have degrees < p and therefore they have the form (|4.10p with 
zero higher order terms. Thus 

<Zsp&n['Y^z^w'*'^V{w)*x : weCxeC^j 
U=o J 
and Da = C £p. This proves Lemma lO □ 
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Proof of the only if statements in Theorem W.W Assume that the reproducing ker- 
nel of 03 is a polynomial Nevanlinna kernel K{z,w): 

(4.11) \{z~w*)K{z,w)^V{z)Q~^'P{w)* for all z,u;eC, 

where Q is a self-adjoint 2d x 2d matrix with d positive and d negative eigenvalues 
and P(z) \s a. d X 2d matrix polynomial with rankP(z) = d for some z S C. Note 
that (|4.1ip implies (jaj) of Lemma 14.21 

(4.12) P(z)Q-ip(z*)* = for ah z G C. 

Wc prove ([A| and (|B| in the steps (i)-(iv), in step (v) we prove the last only if 
statement in the theorem. 

(i) In this step we prove ([B]) under the assumption that (jb| and (jg) of Lemma 321 
hold. Denote by *Bp the reproducing kernel Pontryagin space with kernel 

Lp(z, w) = i (zP - w*P)K{z, w) = z'p-^~^w*^\ P(z)Q'ip(w)*, z,w£C. 

\fc=0 / 

Then 

«Bp = span! P(z)Q^i ^ zP-^-^w*^V{wYx : weC, x€<cA 
I fc=o J 

and 

p-i 

[P(z)Q-i/,P(^)Q"\9]<B^ =^(t'*''^-'-"^P(«)*2/)*Q"'(u'*"P(t")*a;), 

fc=0 

where 

p— 1 p— 1 

f{z) zP-^-^w*^I'{wYx, g{z) = Y zP-^-^v*^V{v)*y. 

Comparing this inner product with the one defined in Lemma 14.21 we find that 
P(z)Q~^ considered as a multiplication operator maps £p C C^''[z]<p isometri- 
cally onto *Bp and its null space is £p (see the second of the three equivalences 
in the beginning of the proof of Lemma [4.21) . Hence, dimSp = 2(0-1 + • • • + (Jd) 
and the positive and the negative index of 58p equal cti + ■ ■ ■ + ad- According to 
Lemma [2.71 wc have dimOS = cti + • • • + (Td- The space *8 is spanned by the columns 
of K{z, w), w £ C and for j £ {1, . . . ,d} the degree of the j-th row of K{z, w) as 
a polynomial in z is equal to max|0,CTj — l}. Therefore *8 C ^j^i{C[z]<^aj)ed.k- 
Since both spaces have dimension ai + ■ ■ ■ + a^, equality prevails: 

d 

(4.13) 55o = 0(C[z]<,^.)erf,fc. 
This implies ([B]). 

(ii) In this step we prove (|X]) under the assumption that (|b| and (jcj) of Lemma l4!2l 
hold. Set 



M(z) = 
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Then Poo = liniz^oc- M{z)P{z) and by ((4T2|) we have 

(4.14) PooQ^^P^o = lim M{z)P{z)Q~'^P{z*)*D{z*)* = 0. 

Since Poo has fuU rank, (|4.14[) imphes that the hnear span of the columns of P^ is 
a maximal neutral subspace of (C^^, [ • , • ]q) and this span coincides with the null 
space of PooQ~^- We claim that for a G C^^ 

(4.15) P(z)ae'B ^ Pooa = 0. 

To prove the claim assume first that P{z)a G From (|4.13p we see that the 
degree of the j-th entry of the vector polynomial P{z)a is strictly less than aj,j e 
{l,...,d}. Hence Pooa = '^^i^z^oo M{z)(P{z)aj — 0. As to the converse, first 
notice that by the definition of Poo the row degrees of the matrix polynomial 

Po{z) = P{z) - M(z)"iPoo are strictly less than ajj e {1 ,d}. By (|jT3)) we 

have that Po{z)a e *8 for all a e C^"*. Now assume PooO = 0. Then 

P{z)a = Po{z)a + M(2)"^Pooa = Po{z)a G «B. 

This completes the proof of (|4.15p . 

Consider / S *B. Since *B is finite dimensional it can be written as 

m 

(4.16) f{z) = K{z, w^)x,, men, G C, x, gC^, m}. 

i=l 

The next sequence of equivalences follows from (|4.15p and the observation after 

m 

<^ ^(z - w*)K{z, Wi)x, e <B 
1=1 

^ P{z)Q-'(^Piw,)*x.}j e'B 

in 

^ 'YP{wi)*Xi =Pl^x for some x e C^. 
Let / g *B be given by (|4.16p and let g e «8 be of the form 

n 

ai.^) = H ^ ^' ^ e C''' j e {1, . . . , n}. 

Assume that f,g £ dom^fg. Then there exist x,y £ such that 

m rn 

Y,P{w^rx^^P*ooX and Y.^{v.,ry.,=P*^y 

i=l 1=1 

and using the reproducing kernel property of K{z,w) we have 

n 7n I m n 

[Sf.a]^ - [f^Sg]^ = YYvjy*K{vj,Wi)x, - YYw,x*K{w^,Vj)yj 
j=i 1=1 \ t=i j=i 
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j—1 i—1 i—1 j—1 

n 7n 

= J2^(^3 ~ W*)y*K{Vj,w{)Xi 

j=l 1=1 

n rn 

= (e y;pK)) (e pK)*-^. j 

= -iy*PocQ-^PL^ 
= 0. 

This proves that 5*03 is symmetric. 

(iii) In this step we only assume (|b| of Lemma [4.21 rankP(z) = d for all z € C. 
Then there is a unimodular dxd matrix polynomial U (z) such that S(z) = ?7(z)P(z) 
is row reduced with ordered row degrees (Xi > • • • > Ud- Then U is an isometry 
from *8 onto the reproducing kernel Pontryagin space £ with kernel 

(4.17) -i"(-)Q-^"(<. 

z — w* 

According to what has already been proved in (i) 

d 

C/*8 = e: = 0(C[z]<,^)ed,fc. 

J=l 

Thus <B = JJ-^t and, by Theorem [331 © holds for all a e C. According to part 
(ii) of this proof, S'c/ijs is symmetric, hence S<s, = U^^ Su'sU is also symmetric, that 
is, © holds. 

(iv) Finally we prove that ([^ and (jBj hold if rankP(z) — d for some z e C as 
in the beginning of this proof. In that case there exist a d x d matrix polynomial 
G(z) with detG'(z) ^ and a d x 2d matrix polynomial S(z) with rankS(z) = d 
for all z G C such that P(z) = G(z)S(z) for all z € C, see Lemma [^ If by 2t we 
denote the reproducing kernel space with Nevanlinna kernel (|4.17p . then, by what 
has been proved in (iii), the operator 5*2 is symmetric and for almost all a g C we 
have ran(S'2i — a) = 21 n keriJ^. Now and ([BI follow since the multiplication 
operator G corresponding to G(z) is an isomorphism from 21 onto 

(v) The last only if statement in the theorem follows from step (iii) above and 
Theorem [331 □ 



Remark 4.3. Assume that *B C C^lz] is a Pontryagin space which satisfies the 
conditions jXI and ([B| of Theorem 11.11 Then, by Theorem 11.11 there is a gen- 
eralized Nevanlinna pair {M(z),A^(z)} such that the d x 2d matrix polynomial 
P(z) = [M(z)A^(z)] provides a representation (|2.7p . with Q given by (|2.8p . for the 
Nevanlinna reproducing kernel K{z,w) of *B. In addition, by Theorem 11.21 there 
is a canonical subspace £ such that *B = W<L for some d x d matrix polynomial 
W{z) with detW{z) ^ 0. The proof of Theorem 1 1.1 1 and Lemma [231 show that the 
multiset of the Forney indices of P(z) coincides with the multiset of the degrees of 
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£. This implies that the Forney indices are independent of the Nevanhnna repre- 
sentation ()2.7p of the kernel K{z, w). In the special case when the defect numbers 
of S<s arc equal to d this fact can also be proved directly by using Theorem 1.3]. 
In view of Remark 14. 1[ this remark substantiates the observations about the Forney 
indices and the defect numbers after Theorem 11.21 in the Introduction. 



5. Q-FUNCTIONS 

5.1. Let M{z) be a generalized Nevanlinna d x d matrix function and denote by 
C{M) the reproducing kernel Pontryagin space with reproducing kernel Km{z, w) = 
KMjdi^^"^)- By [TBJ Theorem 2.1], the operator S in C{M) of multiplication by 
the independent variable is a simple symmetric operator with equal defect numbers 
and its adjoint is given by 

S* ^spMi{{KMi - ,w*)x,'wKm{ - ,w*)x} : a; S C^, w S hol(Af )} 

= {{/,g} e C{Mf : 3x,y'E& such that g^z) - zf{z) = x- M{z)y}. 

It follows that for all w e hol{M) 

ker(S'* -w) = {Km{ ■ ,w*)x : x e C''} = ran£'* , 

where E.i^ is considered as a mapping : C{M) C^. Taking orthogonal com- 
plements we see that 

ran(5' - a) = C{M) n kerS^, a e hol(M). 

Thus (j^ and jBj of Theorem II. II hold. Moreover, [16l Theorem 2.1] and its proof 
imply that there is a constant invcrtiblc d x d matrix T such that 



TM{z)T* = Mo 



M[z) 




where Mq is a constant self-adjoint d x d matrix and, if the defect numbers of S 
are denoted by I, M{z) is a generalized Nevanlinna I x I matrix function which is 
a Q-function for S. The theorem below concerns a converse implication. But first 
we recall the notion of a Q-function. 

Let S* be a simple symmetric operator in a Pontryagin space /C with defect 
numbers equal to I. Let ^ be a self-adjoint extension of S in JC with a nonempty 
resolvent set p{A). Let fi £ p{A) \ R and define a function P^ : C' — > /C such that 
it is a linear bijection from C' onto kcr(S'* — /i). Finally, for z G p{A) define the 
defect mappings Pz : C' — > /C by 

P, = (/ + (z-m)(A-2)-i)P^, zepiA). 

Then P^ is a bijection from C' onto ker(S'* — z), 

(5.1) /C = spiS{P^c : z e /9(A) n (C\R),c e c'} 

and, by the resolvent identity, P^Pz = P**Pmi*, w, z € p{A). A Q-function for S is 
by definition an ^ x ^ matrix function that satisfies the equation 

5.2 =P^,P2, z,w^p[A). 

z ~ w* 

Clearly, Q{z) depends on the choice of the pair {A, P^} and if this choice has to be 
mentioned explicitly we shall say that Q{z) is a Q-function for S associated with the 
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pair {A, r^}. Q{z) is uniquely determined up to an additive constant self-adjoint 
d X d matrix Qq: 

Q{z) = Qo - iImA'r*r^ + {z - ^J.*)T*^Tz, Qo = Qo- 

From (|5.1I) and the defining relation (|5.2p it follows that Q{z) is a generalized 
Ncvanlinna I x I matrix function with k negative squares where k is the negative 
index of the Pontryagin space /C; in particular Q{z)* = Q{z*). Q-functions in an 
indefinite setting were introduced and studied by M.G. Krein and H. Langcr in 

5.2. The following theorem shows that the Ncvanlinna pair {M{z), N{z)} of ma- 
trix polynomials M{z) and N{z) in Thcorcm ll.il can be chosen such that dctN{z) ^ 
and such that N{z)~^M{z) is essentially the Q-function for S<s- As before, by 
C{Q) we denote the reproducing kernel space with reproducing kernel given by 

(ESI). 

Theorem 5.1. Let ^ be a finite dimensional Pontryagin subspace of C^[z] for 
which the conditions (jA]) and (|B]) of Theorem \l.l\ hold. Denote by I G {l,...(i} 
the equal defect numbers of the symmetric operator S<s- Let Q{z) be an I x I 
matrix Q-function for S<s- Then there is a d x d matrix polynomial N{z) with 
detA^(2;) ^ such that M{z) = N [z)Aia,g{Q{z) ^ 0) is a dx d matrix polynomial and 
% = N {C{Q) {0}). In particular, {M{z), N{z)} is a Ncvanlinna pair of matrix 
polynomials and Ki\i^n(z,w) is the reproducing kernel of^. 

Proof. Assume and ([B]) of Theorem 11.11 By Theorem 11.21 there is a d x d 
matrix function W{z) with detW(z) ^ such that *B = W€, where £ is a canonical 
subspace of C''[z]. By Remark 14.11 the defect numbers of the symmetric operator 
S<s are both equal to I with I < d. Wc consider two cases: I = d and / < d. 

(i) I = d. Let Q{z) be the Q-function for S<s associated with the pair {A, Fz}, 
where A is a self-adjoint extension of S<s, and the defect mappings are defined 
above with I = d. Since S<s, is simple, the mapping 

f^g with f{z) ^ F:.F„. X, g{z) = (F^.x) (z), x € £.\ w € p{A), 

can be extended by linearity to a unitary mapping U from C{Q) onto *B. That U 
is isometric follows from 

\^*w^zX,TlV^y]^^^^ = y*^l,^vx = j/*F*.F„,. x = [F„,.a;, F„.y]>B, x,y e C^. 

We claim that U is the operator of multiplication by a d x d matrix function. To 
prove the claim we use the equality *B = W€. Since the defect numbers of S<s are 
equal to d, the degrees of £ arc all > 1 (see Remark |4.1|) and hence the d columns 
of W{z) belong to *B and are linearly independent over C. We denote by F** the 
d X d matrix function defined by 

(F*. W)x = F*. (Wx), a; e C^, z G p{A). 

We show that its inverse exists for z e p{A) n {z G C : detW(z) ^ 0}. 

Suppose there is an a; e such that (F*.M^) x = 0. Then for all y e 

o^[{r:,w)x,y]^,^[r:,{Wx),y]^, = [Wx,r,,y]^, 

hence Wx € ker(S'^ - z*)^ = ran(S'<8 - z) = ^ n E^. That is, W{z)x = and it 
follows that a; = 0. This proves that (F*.VF)-i is well defined for aU z e 17. Wc set 
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N{z) = W{z){ri.W)-'^. Clearly, det7V(z) ^ 0. We have shown that U coincides 
with multiplication by N(z) if we have proved that 

N{z)Tl,T^,x = (r„.x) (z), X e we p{A), zen, 

or, equivalently, that with y{z, w, x) := W{z)^^ {T^»x) (z) g 

ri,r^,x = Tl, {Wy{z,w,x)) , xgC^, wep{A), zen. 

But this equality holds, since F** (ran(S'(8 — z)) = {0} and 

Tujtx — Wy{z, w,x) <E ^ C\ Ez ~ ran(S'(B — z). 

This completes the proof of the claim that U is multiplication by N{z). It follows 
from [6l Theorem 1.5.7] and its proof that the formula for the kernel K{z^ w) of *B 
is given by 

Kiz,w)^Niz)9^^^Niwr 

and hence «8 = NC{Q). 

It remains to show that M{z) = N{z)Q{z) and N{z) are matrix polynomials. 
Since the elements of the space *B are polynomials, the matrix function z i— > K{z, w) 
is a matrix polynomial, hence the matrix function 

M{z) - N{z)Q{wy = N{z)Q{z) - N{z)Q{wy = (z - w*)K{z,w)N{w)~* 

is a matrix polynomial in z. Thus if N{z) is a matrix polynomial, then so is M{z). 
It remains to show that N{z) is a polynomial. For this we note that the above 
formula implies that for x £ 

Qi^n-QH* (z - fi*)K{z, ^^)N{^l)-* - (z - w*)Kiz, w)Niwr* 

JM (z) X = X. 

/i* — W* /I* — w* 

The right hand side is a matrix polynomial in z and hence it follows from the 
equality that N{z) is a matrix polynomial if we can show that 

& = span{F*F„.a; : w E p{A) n (C \ M), xe 

To prove this equality we argue by contradiction and suppose it is not true. Then 
there is a nonzero vector x £ orthogonal to the set on the right hand side, that 
is, 

[F^.y,F^x]^ =0, we p{A)n{C\R), yeC". 

Since S<s is simple and F^ is injective, we find that T^x = and that x = 0, which 
contradicts the choice of the nonzero vector x. 

(ii) I < d. Then £ = £i {0}, where £i is a canonical subspace of C'[z] of which 
the degrees are all > 1. Using the relation *B = W © {0}) we equip d with 
an indefinite inner product that makes W an isomorphism. Then S<b and Sc-^ are 
isomorphic: S'c^ = WS<s,W^^ , hence S^-^ is symmetric and has defect numbers 
equal to I. Thus (|^ holds and it is not difficult to verify that also ([B]) holds on 
£i. Finally, since Q{z) is the Q-function for S<s associated with the pair {^,r2}, 
Q(z) is the Q-function for S^^ associated with the pair {ly^^^VF, M^^^F^}. This 
all shows that we may apply part (i) of this proof (with W{z) = Ii): There exists 
axilxl matrix polynomial -/Vi(z) with detiVi(z) ^ such that Ni{z)Q{z) is an ^ x ^ 
matrix polynomial and £i = Ni£{Q). It follows that if 

7V(z) = iy(z)diag(7Vi(z),/d_0, 
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then detN{z) ^ 0, N (z) diag{Q {z) , 0) is a d x d matrix polynomial and *8 = 
N{CiQ)®{0}). □ 

6. Corollaries and examples 

In the next corollary we extend Theorem 11.11 to finite dimensional Pontryagin 
spaces of rational vector functions. A rational Nevanlinna kernel is a kernel of the 
form Km,n{z,'w) as in (|1.6|) . in which M{z) and N{z) are rational d x d matrix 
functions satisfying (|1.4p and (jl.Sp . 

Corollary 6.1. Let ^ be a finite dimensional Pontryagin space of rational d x 1 
vector functions and letQcC be the finite set of all the poles of the functions in *8. 
Denote by S<s the operator of multiplication by the independent variable in *B and 
by Ea the operator of evaluation at a point a € C Then the reproducing kernel of 
*B is a rational Nevanlinna kernel if and only if the following two conditions hold: 

(a) The operator Sfs is symmetric in S. 

(b) For some a G C \ we have ran(S'(B — a) = 58 H ker Ea- 

Proof. Assume (jsj) and ([b|. Let q{z) be the monic scalar polynomial of minimal 
degree such that *B' := {q{z)f{z) : f e *B} consists of polynomials. Equip *B' 
with the Pontryagin space inner product that makes the mapping g : *B — > 58' of 
multiplication by q{z) a unitary mapping. Then items © and (|B|) of Theorem O 
hold for 58'. Hence *B' has a polynomial reproducing Nevanlinna kernel Km,n{z^ w). 
It follows that *8 has reproducing kernel Kj^[/qj^/q{z,w). 

Now assume Km,n{z,w) is a rational reproducing Nevanlinna kernel of 58. Let 
r(z) be a polynomial such that r(z)M{z) and r(z)N{z) are polynomials and hence 
form a polynomial Nevanlinna pair {r{z)M (z), r{z)N(z)}. Then KrM,rN{z, w) is a 
polynomial reproducing Nevanlinna kernel of the space 58" := {r{z)f{z) : f{z) e 
5B} equipped with the inner product that makes multiplication by r(z) an isomor- 
phism from 58 onto 58". Since the elements of 58" are polynomials, we can apply 
Theorem 11.11 to conclude that items (j^ and (|BJ hold for the space 58". Since 
multiplication by r{z) and by z commute, ((A|) implies (jaj). By Theorem 11.21 and 
(|1.3p . the equality 



(6.1) ran(S'<8" - a) = S" n kcr£;„ 

holds for all but finitely many a £ C Choose a G C \ such that (|6.ip is valid. 
Then for this a item (jbj holds. □ 

Corollary 6.2. Let (58,[-, -Jts) be a finite dimensional Pontryagin subspace of 
C^lz] whose reproducing kernel is a Nevanlinna kernel determined by a generalized 
Nevanlinna pair. Let J be a fundamental symmetry on 58 . Then the Hilbert space 
(5B, [J ■ , ■ ]<s) has a reproducing Nevanlinna kernel determined by a Nevanlinna pair 
if and only if S<s is symmetric in this space. 

The corollary follows from Theorem 11.11 because condition jB]) is independent 
of the topology on 58. 

Example 6.3. Consider the subspace 58 of C^[z] spanned by the columns of the 
matrix 

ri z z2 0" 

^^^'>~ 1 
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and equipped with the inner product 



]<s so that 



G 



is the Gram matrix associated with B{z): G = [B, B]<s 
of G is G = UJU* with unitary matrix 

\/2 

-110 

V2 

110 



The spectral decomposition 



V2 



and J = 



It follows that (*B, [ • , • ](b) is a Pontryagin space with positive index 3 and negative 
index 1. The equality [BU, BU]<s = J defines a fundamental decomposition of *B 
with corresponding fundamental symmetry determined by SfBU = BUJ. In the 
Hilbert space inner product [•, [J' ■ , ■]<b we have [BU, BU]j ~ ~ 

and hence [B,B]j = I„. The operator S<s is symmetric in the Pontryagin space 
(Q3,[-, -Js), but not in the Hilbert space Since 58 is a canonical 

subspace of C^[z]. Theorem 11.21 implies that Theorem 11.11 (| B |) holds in S. Hence, 
according to Theorem 11.11 the Pontryagin space (*8, [•, -Jos) has a reproducing 
Nevanlinna kernel, whereas the reproducing Hilbert space (23,[-, does not 
have a reproducing Nevanlinna kernel. □ 

Corollary 6.4. Let S be a finite dimensional Pontryagin subspace of C^[z] whose 
reproducing kernel is a Nevanlinna kernel. Let *Bo be a Pontryagin subspace of^. 
Then the reproducing kernel of So is a Nevanlinna kernel if and only if for some 
a £ C we have ran(S'f8Q — a) = *Bo H ker i?Q,. 

The corollary follows from Theorem II. 1[ because the hypothesis implies that 
S'fgg, being a subset of S<s,, is symmetric in *Bo, that is, that ([5]) holds for S<s,a- 

Example 6.5. Consider the Hilbert subspace 

So = span ■ 







Then for arbitrary a G C we have 
and *Bo n ker Ea ~ span 



of the space *B in Example 

ran(S'<8o - a) = {0} 

Thus the condition ran(S'<8o ^'^) ^ 25onkeri?Q does not hold for any a £ C. Hence 
Corollary implies that the reproducing kernel of *Bo, which is calculated to be 

"l + z^w*^ 



K{z, w) 







is not a Nevanlinna kernel. This fact can be verified using 
observe that for all z, w € C we have 



Theorem 1.3]. First 



{z - w*)K{z, w) = M{z)N{w)* ~ N{z)M{w)* = [M{z) N{z)\ 



N{w)* 
-M{w)* 
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(6.2) 



{z~w*)K{z,w) - [Mi(z) 7Vi(z)] 



z,w £ 



where 

iV(z) = Q Q and M(z) = ziV(z), z e C. 

Now |231 Theorem 1.3 and Section 4] imply that for any 2x2 matrix polynomials 
Mi{z) and Ni{z) such that 

Ni{w)*' 
-Mi{w)* 

there exists a 4 x 4 invertible matrix S such that 

[Mi{z) Ni{z)] = [M{z) N{z)]S, zeC. 

Hence (|6.2p yields that rank[Mi(z) -/Vi(z)] = 1 for all z £ C. Consequently 
K{z, w) is not a Nevanlinna kernel. Using the same results from [23] one can also 
show that the scalar reproducing kernel K{z,w) = 1 + z'^w*'^ of the Hilbert space 
with orthonormal basis {1, z^} is not a Nevanlinna kernel. □ 

We end the paper with two examples in which det i5r(z, w) = 0. These examples 
also show that the proof of Theorem 15.11 is constructive. 



Example 6.6. Consider the space *B with reproducing kernel 

"0 -1 

IC{z,w)= -w* 
-1 -z 

We show that, even though deiK{z,w) = 0, the kernel is a Nevanlinna kernel. We 
follow the proof of the first part of Theorem 15 . 1 1 and construct two Nevanlinna pairs 
that determine K(z, w). The space 05 is spanned by the columns of the 3x4 matrix 
polynomial 

[1 0' 
B{z) =0100. 
1 z 

It follows that *8 is a canonical subspace of C^[z] with degrees 1, 1 and 2. The 
Gram matrix associated with B{z) is given by 

-/ 



G=[B,B]<s 



-I 



hence 05 is a Pontryagin space with positive and negative index 2. The operator of 
multiplication by z on *B is given by 



S<B — 



B 



a e 



It is easy to see that and ([B|) of Theorem 1 1.1 1 are satisfied. The defect numbers 
of S<s are both equal to 3, see Remark l4.ll The Q-function of S^s associated with 
the self-adjoint extension 



A = 







a 




"0" 













c 




II 





> 




b 













A 





a,b,c,d(l C 
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of S<s (which is muhi-valued and has a nonempty resolvent set) and the defect 
mappings Tz ^ ker{S* — z) defined by 



{l+{z-i){A-z)-^)T, 



B 



i/z 
I 









i/z 

1 



IS 



Q{z) = Qo 



1/z iz 
1/z 
-iz 



Qo — Qo- 



Wc find that X{z) = Y{z)Q{z) and Y{z) = (r*.I) ^ form a fuU gcncrahzcd Nevan- 
finna pair of matrix polynomials: 



X{z) = Y{z)Qo + 



-i 
iz 
-i 



and Y{z) = 



-iz 
0-1 
-iz 



such that K{z,w) = Kx,y{ztW) and [^(z) i^(z)] is row reduced with Forney 
indices 1, 1 and 2, which is in accordance with Remark 14.31 

The Q-function associated with the self-adjoint operator extension A of S<s and 
defect mappings T z defined by 



AB = B 



"0 


1 





0" 




















, Tz = B 








1 



























i/z 








-(z-i)/z2 


i/z 



is given by 



Q{z) = Oi + 



iz^ + z + i 

i 

z 



z^ 






— 1 
z 






Qi = Ql. 



Again we find that M[z) = N{z)Q[z) and N{z) = (r*.I) arc matrix polynomials: 

— iz z + i 



Af (z) = N{z)Qi + 



z 

1 

— iz^ + z — i — iz 



, N{z) = 







— iz 




which form a full generalized Nevanlinna pair such that K{z, w) ~ Km,n{z, w) and 
[M(z) -/V(z)] is row reduced with Forney indices 1, 1 and 2. □ 

Two generalized Nevanlinna pairs {X{z), Y{z)} and {M{z), N{z)} of dxd matrix 
polynomials define the same Nevanlinna kernel if one is a J-unitary transformation 
of the other, that is, if they are connected via the formulas 

M(z) = X{z)A + Y{z)C, N{z) = X{z)B + Y{z)D, 

where A, B, C and D arc constant d x d matrices such that if wc set 



U = 



A B 
C D 



and J = 





-ild 



ild 
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then U is J-unitary: UJU* = J. The pahs {X{z),Y{z)} and {M{z),N{z)} in Ex- 
ample [6l6] (with arbitrary constant self-adjoint matrices Qo and Qi) are connected 
via a J-unitary transformation. The following example shows that the converse of 
the foregoing statement does not hold. 

Example 6.7. Consider the Nevanlinnna pair {X{z),Y{z)} given by 

X{z) = diag(z, 0, z^), Y{z) = diag(0, z, z). 

Then the space 03 with reproducing kernel Kx,y{^j''^) = diag(0, 0, zw*) is a 1- 
dimensional Hilbert space: it is spanned by B{z) = [O z]^ and the corre- 
sponding Gram matrix is G = [B,B]<:g ~ 1. Note that detX(z) = 0, detF(z) = 
and P(z) := [X{z) Y{z)] does not have fuU rank at z = 0: P(0) = 0. We show 
that the pair {X{z),Y{z)} can be replaced by a Nevanlinna pair {A/(z), N{z)} such 
that det7V(z) ^ 0. 

Since P(z) does not have full rank for all z g C, to calculate the Forney indices 
we must first apply Lemma [2T4l We write P(z) as P(z) = G(z)T(z) with 





"0 





z 




"0 





z 








1' 


G(z) = 





z 





, T(z) = 














1 







z 










1 


















Since T(z) has full rank for all z e C and is row reduced, the Forney indices of 
P(z) are those of T(z) and they arc /ii — 1, fJ.2 = f^s = 0. This fits in well with the 
observations after Theorem 11.21 indicating that dim *B = 1 and the defect numbers 
of the symmetric operator S<s — {{0,0}} are both equal to 1. We follow part (ii) 
of the proof of Theorem [O and write S = (£i ® {0}) with 



W{z) 



1 
1 
z 



and £i = C. Wc make the multiplication operator W an isometry when C is 
equipped with the Euclidean inner product. Then S'cj = {{0, 0}} is symmetric. The 
defect subspaces ker(S'g — z) all coincide with C and A is a self-adjoint extension 
of if and only ii A = Am, the operator of multiplication by m, m S R, or 
A = A^ci ~ {{0,c} : c G C}. Since £i is a Hilbert space, all self-adjoint operators 
and relations have a non-empty resolvent set. Choose /iGC\R, 7GC \ {0} and 
define : C — > ker(S'J^ — /i) = C by F^a; = jx, a; G C. Then the Q-function g(z) 
of S'cj associated with {A, F^} is given by 



|j71-^|2|7p 



q{z) = qo + 



|7pz 



if ^ — Ami 

if A = A,ci, 



where qo is an arbitrary real number, and 

m — z 



/ \" if A — Am , 

7*(m — ji*) 

^ if A = A-el. 
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We find that Kx,y{z,w) = Km,n{z,w) with matrix polynomials 



M(z) = N{z) 














and N{z) = W{z) 



(r:.i)- 








1 
1 



It is easy to see that the Ncvalinna pairs {X(z),y(z)} and {M{z),N{zy\ are not 
related via a J-unitary transformation. □ 
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